Strengthening the feedback culture in a postgraduate residency program.

Background: Feedback is defined as specific information presented to a learner that facilitates professional development through the process of reflection. Timely provision of constructive feedback to learner is important in optimizing the learning curve. The aim of the current study was to see the effectiveness of various interventions on feedback practices of faculty members. Methods: This is a quasi-experimental study (pre- and postdesign). It was conducted from November 2009 to March 2011 at The Aga Khan University, Pakistan. Faculty development workshops, allotment of specified feedback time, and restructuring of residency feedback forms were done as interventions. Data collection was done pre- and postintervention. Resident's and faculty satisfaction regarding the feedback process were evaluated using a prepiloted questionnaire. Paired t-test was applied to assess the effect of interventions on faculty and resident's satisfaction. Results: The mean satisfaction scores of residents were significantly improved (P < 0.05). Pre- and postintervention faculty satisfaction score also demonstrated significant difference in overall satisfaction level, from 47.88 ± 13.92 to 63.40 ± 8.72 (P < 0.05). Discussion: This study showed improved faculty engagement and satisfaction for the provision of feedback to the trainee resident. Strengthening this, culture requires continuous reinforcement, individualized feedback to the faculty members regarding their feedback practices, and continuing faculty development initiatives.

An Investigation of the Sources of Measurement Error in the Post-Encounter Written Scores from Standardized Patient Examinations.

Purpose. Post-encounter written exercises (e.g., patient notes) have been included in clinical skills assessments that use standardized patients. The purpose of this study was to estimate the generalizability of the scores from these written exercises when they are rated by various trained health professionals, including physicians.Method. The patient notes from a 10 station clinical skills examination involving 10 first year emergency medicine residents were analytically scored by four rater groups: three physicians, three nurses, three fourth year medical students, three billing clerks. Generalizability analyses were used to partition the various sources of error variance and derive reliability-like coefficients for each group of raters.Results. The generalizability analyses indicated that case-to-case variability was a major source of error variance in the patient note scores. The variance attributable to the rater or to the rater by examinee interaction was negligible. This finding was consistent across the four rater groups. Generalizability coefficients in excess of 0.80 were achieved for each of the four sets of raters. Physicians did, however, produce the most dependable scores.Conclusion. There is little advantage, from a reliability perspective, in using more than one trained physician, or other health professional who is adequately trained to score the patient note. Measurement error is introduced primarily by case sampling variability. This suggests that, if required, increases in the generalizability of the patient note scores can be made through the addition of cases, and not the addition of raters.